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1.  INTRODUCTION:  The  proposed  research  is  focused  on  restoration  of  hand  motor  and  sensory 
functions  by  utilizing  a  direct  electrical  interface  with  residual  peripheral  nerves.  The  direct 
connection  with  the  residual  nerves  will  enable  the  patient  to  have  intuitive  control  over  and 
receive  touch  sensation  from  a  prosthetic  hand  that  are  not  provided  by  current  forearm 
prostheses.  The  improvement  in  intuitive  control  and  the  providing  of  sensory  feedback  will 
allow  patients  to  use  highly  articulate  prosthetic  hands  with  improved  long-term  functional 
outcomes  for  military  personnel  and  civilians  with  a  forearm  amputation.  Based  on  preliminary 
studies  in  peripheral  nerves  it  is  possible  to  decode  finger  movements  from  electrophysiological 
signals  recorded  from  peripheral  nerves,  and  to  evoke  somatosensory  perceptions  through 
micro-stimulation  of  peripheral  nerves.  The  proposed  research  will  determine  the  type  and 
complexity  of  movements  that  can  be  controlled  by  a  direction  connection  to  arm  nerves,  and 
will  also  determine  the  type  and  range  touch  sensations  that  can  be  provide  through  a  direct 
connection  to  residual  nerves. 

2.  KEYWORDS:  Peripheral  Nerve  Interface,  Prosthetic  Hand,  Neural  Prosthesis,  Sensory  Feedback, 
Micro-stimulation,  Electrophysiology,  Action  Potentials,  Micro-electrode,  poly-Longitudinal 
Intrafascicular  Electrode  (poly-LIFE) 

3.  ACCOMPLISHMENTS:  Provided  in  italics  in  appropriate  sections. 

What  were  the  major  goals  of  the  project? 

Specific  Aim  1:  Dexterous  control  of,  and  sensory  feedback  from,  an  advanced  prosthetic 
hand  will  be  provided  using  micro-electrode  arrays  implanted  in  residual  peripheral  nerves 

Major  Goal  1:  Preparation  for  Studies 

Subtask  1.1:  Regulatory  Approvals  (Months  1-4) 

•  Mayo  Clinic:  Dr.  Shelley  Noland  and  Nicole  Cevette  oversee  writing  and  submission 
of  protocol  and  associated  documents  for  IRB  and  HRPO. 

•  Dr.  Greger  will  assist  in  writing  and  submission  of  IRB  and  HRPO  documents. 

90%  complete. 

Study  protocol  approved  by  Mayo  Clinic  IRB  on  5-Aug-2016. 

ASU  IRB  confirms  Mayo  Clinic  as  external  IRB  of  record  on  10-Aug-2016. 

Subtask  1.2:  Micro-electrode  Arrays.  (Months  4  -  30) 

•  Production  of  9  human-ready  Sputtered  Iridium-Oxide  (SIROF)  micro-electrode 
arrays  consisting  of  100  electrodes. 

50%  complete.  Design  and  drawings  of  micro-electrode  arrays  complete  and 
purchasing  quotes  obtained  from  Blackrock  Microsystems. 

•  Measure  impedances  on  all  electrodes  in  each  array  prior  to  sterilization,  and  if 
possible  after  explantation  at  the  end  of  the  study. 
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50%  complete.  Instrumentation  to  obtain  impedances  on  micro-electrode  arrays  at 
multiple  frequencies  currently  in  place  and  validated. 

•  Obtain  light  microscope  images  of  the  arrays  prior  to  implantation  and  after 
explantation. 

30%  complete.  Contract  in  place  to  perform  light  and  electron  microscopy  on  the 
micro-electrode  arrays  with  the  Aberration  Corrected  Electron  Microscopy  core 
facility  atASU.  Have  obtained  test  images. 


Major  Task  2:  Implantation  and  explantation  of  micro-electrode  arrays  in  a  residual  nerve 
of  patients  with  trans-humeral,  trans-radial,  or  elbow  disarticulation  amputations 

Subtask  2.1:  Patient  Recruitment  (Months  4  -  30) 

•  Volunteers  will  be  recruited  and  informed  consent  obtained  at  the  Mayo  Clinic  using 
the  procedure  in  the  approved  IRB  protocol. 

•  We  expect  to  recruit  6  patients  (~2/year)  for  participation  in  the  study. 

10%  complete.  Patient  recruitment  materials  developed  and  approved  by  Mayo 
Clinic  IRB.  Patient  recruitment  will  not  begin  until  protocol  is  approved  by  HRPO. 

Subtask  2.2:  Micro-electrode  array  implantation  (Months  4  -  30) 

•  Implantation  of  one  micro-electrode  array  in  either  the  median,  radial,  or  ulnar 
nerve  of  each  patient  will  be  performed. 

Milestone  #1:  Implantation  of  first  patient.  (Months  6) 

Subtask  2.3:  Micro-electrode  array  explantation  (Months  9  -  36) 

•  Explantation  of  the  micro-electrode  array  will  be  performed  at  the  completion  of 
the  study  (30  -  90  days  post-implantation). 

0%  complete.  We  cannot  implant  or  explant  first  patient  until  study  protocol  is 
approved  HRPO. 


Major  Task  3:  Recording  of  isolated  action  potentials  or  multi-action  potential  activity 
from  residual  peripheral  nerve  while  patient  intends  movements  of  amputated  hand/arm 

Subtask  3.1:  Mapping  of  neural  activity  (Months  4  -  36) 

•  Patients  will  be  asked  to  intend  a  number  of  individual  finger  and  multiple  finger 
flexion,  extension,  adduction,  and  abduction  movements  of  their  amputated  hand 
by  mimicking  computer  controlled  movements  of  the  virtual  prosthetic  hand. 
Similarly,  they  will  be  asked  to  make  pronation-supination  forearm  movements;  and 
flexion,  extension,  adduction,  and  abduction  movements  of  their  wrist. 
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•  The  spatio-temporal  patterns  of  action  potential  firing  evoked  in  the  efferent  fibers 
of  the  nerve  will  be  recorded  with  the  micro-electrode  array  during  these  intended 
movements.  We  will  map  the  different  intended  movements  onto  the  neural 
activity  recorded  on  the  electrodes  of  the  micro-electrode  array. 

Milestone  #2:  chronic  electrophysiological  recording  from  first  patient  (Month  6) 

25%  complete.  The  infrastructure  for  performing  the  electrophysiological  recordings  and 
nerve  mappings  is  in  place  and  validated. 

Subtask  3.2:  Offline  analysis  and  decoding  of  movements  (Months  6  -  36) 

•  Analyze  the  data  recorded  during  sub-task  3.1  to  determine  if  the  neural  activity  can 
accurately  predict  the  movements  being  intended. 

50%  complete.  The  workstation  computer  and  data  server  have  been  setup  in  the 
Goldwater  Computing  Center,  and  the  offline  analysis  code  have  been  implemented. 

Subtask  3.3:  Online  (real-time)  decoding  of  movements  (Months  6  -  36) 

•  Using  data  collected  during  subtask  3.1  we  will  train  an  online  decode  algorithm  and 
then  provide  the  patient  real-time  control  over  the  virtual  prosthetic  hand. 

Milestone  #3:  real-time  control  of  multiple  degree  of  freedom  virtual  prosthetic  had  in 
first  patient  (Month  9) 

50%  complete.  Computers  for  real-time  computer  analysis  and  control  of  the  virtual 
prosthetic  hand  have  been  implemented  in  the  patient  cart.  Real-time  data  acquisition, 
analysis,  and  control  of  the  virtual  prosthetic  hand  have  been  validated. 


Major  Task  4:  Evaluation  of  somatosensory  perceptions  evoked  by  electrical  micro¬ 
stimulation  of  the  implanted  nerve  via  the  micro-electrode  array 

Subtask  4.1:  Topographical  mapping  and  subjective  description  of  evoked  perceptions 

(Months  6-36) 

•  We  will  perform  electrical  micro-stimulation  at  various  currents  on  each  electrode  in 
the  micro-electrode  array  in  order  to  determine  the  minimum  current  needed  to 
consistently  evoke  a  sensory  perception. 

Subtask  4.2:  Spatial  two-point  discrimination  (Months  6  -  36) 

•  Using  super-threshold  micro-stimulation  levels  obtained  in  subtask  4.1,  we  will 
determine  if  micro-stimulation  on  pairs  of  electrodes  with  differing  inter-electrode 
spacing  evoke  a  single  perception  or  two  spatially  distinct  perceptions. 

Subtask  4.3:  Modulation  of  evoked  perceptions  (Months  6  -  36) 

•  We  will  determine  if  changes  in  micro-stimulation  parameters  result  in  modulation 
of  the  evoked  sensory  perceptions.  We  will  provide  modulation  of  micro-stimulation 
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frequency  using  input  from  tactile  sensors  such  as  would  be  used  in  a  prosthetic 
hand,  i.e.  micro-stimulation  frequency  would  be  modulated  by  the  amount  of 
pressure  on  a  fingertip  tactile  sensor. 

Milestone  #4:  Completion  of  studies  and  longitudinal  analysis  in  all  6  patients. 

Preparation  of  manuscript  for  publication  in  scientific  literature  (Month  36) 

25%  complete.  The  infrastructure  for  performing  micro-stimulation  to  perform 

electrophysiological  recordings  and  nerve  mappings  is  in  place  and  validated. 

What  was  accomplished  under  these  goals? 

Our  most  significant  progress  has  been  on  obtaining  the  IRB  and  HRPO  approvals  of  the 
study  protocol.  Additionally,  we  have  been  making  progress  on  activities  and  objectives 
related  to  1)  installing  and  validating  the  human  patient  care  for  data  acquisition  and  micro¬ 
stimulation,  2)  improving  the  virtual  reality  prosthetic  hand  and  patient  environment  and 
decoding  algorithms,  and  3)  improving  the  micro-electrode  neural  interface. 

We  have  implemented  the  human  patient  cart  for  performing  electrophysiological 
recordings  and  micro-stimulation  stimulation.  The  patient  cart  has  been  assembled  and  we 
have  performed  the  systems  integration  on  the  computers  and  equipment  needed  for 
performing  the  recording  and  stimulation  experiments.  The  patient  cart  contains  all  the 
hardware  needed  to  perform  the  proposed  real-time  virtual  reality  prosthetic  hand  control 
and  sensory  feedback  experiments.  The  cart  has  been  inspected  and  approved  by  the 
Biomedical  Engineering  department  at  the  Mayo  Clinic  for  recordings.  We  are  in  the  process 
of  identifying  specific  rooms  at  the  Mayo  Clinic  and  at  ASU  that  can  house  this  equipment 
and  be  used  for  the  experiments  with  the  patients. 

We  have  implemented  and  improved  the  virtual  reality  environment  for  patient  training  and 
real-time  control  of  the  virtual  prosthetic  hand.  Patients  in  an  earlier  study  reported  a 
mismatch  between  the  perceived  position  of  their  phantom  hand  and  the  visual  position  of 
the  virtual  reality  prosthetic  hand.  This  interfered  with  their  ability  to  control  the  virtual 
reality  hand  and  their  embodiment  of  the  prosthetic  hand.  In  order  to  improve  patient 
control  during  real-time  decoding  of  neural  signals  and  patient  embodiment  of  the 
prosthetic  hand  we  have  improved  the  virtual  reality  interface.  We  have  made  the  virtual 
reality  prosthetic  hand  environment  fully  immersive  using  Occulus  Rift  goggles.  This 
immersive  environment  allows  the  patients  to  control  a  virtual  reality  hand  mapped  onto 
their  residual  arm.  During  training  the  control  of  the  virtual  prosthetic  hand  is  provided  by 
tracking  movement  of  the  patient's  intact  hand.  The  movements  of  the  intact  hand  can  be 
monitored  by  an  infra-red  tracker  that  is  used  in  a  portable  system  that  can  be  taken  home 
by  the  patients  from  practice  prior  to  array  implantation.  By  using  tracking  of  the  patients' 
intact  hand  prior  to  implantation  of  the  electrode  array  we  will  be  able  to  establish  an 
accurate  mapping  of  the  visual  input  of  the  position  of  the  prosthetic  hand  onto  the  felt 
position  of  the  patient's  phantom  hand.  Additionally,  practice  with  the  virtual  prosthetic 
hand  immersive  virtual  reality  environmental  will  enable  patients  to  "re-activate"  their 
phantom  limb  prior  to  array  implantation  and  therefor  increase  the  likelihood  or  recording 
neural  activity  correlated  with  hand  control.  After  the  micro-electrode  arrays  have  been 
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implanted  and  during  real-time  decoding  experiments,  hand  position  will  be  monitored 
using  a  high-speed  and  high-resolution  motion  capture  system,  which  will  allow  precise 
mapping  of  the  neural  signals  onto  finger  and  hand  position  (Figure  1). 


Figure  1  -  The  infrared  motion  tracking  system.  A  student  is  shown  wearing  the 
Occulus  Rift  goggles  and  infrared  hand  tracking  system.  The  screen  shows  the 
position  of  the  student's  hands  captured  by  tracking  systems  and  displayed  to  her  by 
the  googles.  This  demonstration  of  the  real-time  hand  tracking  and  virtual  reality 
environment  was  filmed  for  use  on  multiple  news  programs. 

We  have  improved  on  past  decoding  efforts  using  machine  learning  algorithms  based  on 
multivariate  time  and  frequency  domain  features.  Patients  and  rehabilitation  specialists 
have  informed  us  that  very  high  performing  decoding  algorithms  are  critical  as  even  a  few 
percent  error  translates  into  an  unacceptable  failure  rate  in  object  manipulation,  e.g. 
dropping  a  utensil  5%  of  the  time.  The  machine  learning  algorithms  are  being  implemented 
on  the  patient  cart  for  use  in  the  real-time  control  of  the  virtual  prosthetic  hand 
experiments. 

Multiple  degrees  of  freedom  were  decoded  using  peripheral  neural  signals  in  an  offline 
implementation  of  a  new  decoding  model.  The  decoding  model  utilizes  a  hybrid  approach  by 
incorporating  both  classification  and  regression  algorithms.  Classification  results  produced 
the  discrete  control  signals  which  were  used  as  pre-cursors  in  selecting  the  correct 
individual  regression  model  to  predict  the  movement  trajectory.  The  previous  study  aimed 
at  creating  a  "global  model"  for  multiple  degrees  of  freedom  using  a  Kalman  filter.  The 
degradation  in  performance  in  higher  degrees  of  freedom  in  the  previous  study  was 
overcome  by  adopting  an  explorative  approach  which  had  an  individual  regression  model 
for  each  degree  of  freedom,  therefore,  not  compromising  on  the  decoding  quality.  The 
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newly  implement  machine  learning  ensemble  method  algorithm  exhibited  reduced  error 
and  improve  stability  of  none  movement  states,  i.e.  it  didn't  indicate  that  a  finger  was  being 
moved  when  the  patient  was  holding  still  (Figure  2). 

Data  from  an  earlier  human 
study  reveal  challenges  with 
using  rigid  silicon  arrays  in  an 
active  human  patient 
population.  The  rigid  silicon 
arrays  were  capable  of 
making  good 
electrophysiological 
recordings  and  performing 
micro-stimulation,  however, 
they  are  susceptible  to  crush 
damage  (Davis  et  al.  Journal 
of  Neural  Engineering  2016). 

We  have  designed  a 
compliant  poly-LIFE  micro¬ 
electrode  array  using  Kevlar- 
fiber  and  cracked-gold 
materials  that  are  similar  to 
electrode  arrays  used 
successfully  in  previous 
human  and  animal  studies 
that  is  resistant  to  crush  damage.  In  contrast  to  the  rigid  silicon  based  electrode  arrays,  the 
materials  used  in  the  poly-LIFE  electrode  arrays  allow  them  to  compliantly  move  with  the 
peripheral  nerves  and  resist  mechanical  damage.  We  have  submitted  and  SBIR  proposal 
through  the  NIH  to  specifically  develop  the  compliant  poly-LIFE  electrode  arrays. 

Although  the  rigid  silicon  arrays  appear  to  have  a  very  challenging  design  issued  due  to  their 
rigidity  and  brittleness,  they  may  be  appropriate  for  use  in  acute  studies  in  a  few  patients  in 
order  to  establish  important  prosthetic  control  and  sensory  feedback  design  parameters. 

The  use  of  compliant  micro-electrode  arrays  will  likely  address  this  issue  and  provide  path 
forward  to  long-term  patient  peripheral  nerve  interfaces.  As  the  compliant  poly-LIFE 
electrode  arrays  become  available  we  will  need  make  addendums  to  the  study  protocols 
approval  through  the  Mayo  Clinic  IRB  and  HRPO. 

What  opportunities  for  training  and  professional  development  has  the  project  provided? 

All  of  the  students  involved  in  the  project  have  had  extensive  training  and  hands  on 
experience  on  implementing  the  hardware  (systems  integration)  and  the  software  for  offline 
analysis  and  real-time  decoding  and  stimulation.  They  have  had  experience  performing 
intraoperative  electrophysiological  neural  recordings  in  human  patients.  They  have  been 
participating  in  all  of  the  meetings  with  Mayo  Clinic  physicians,  surgeons,  and  hospital  staff. 


Post-implantation  day 


Figure  2  -  A  significant  reduction  in  the  mean 
squared  error  was  achieved  using  a  machine  learning 
ensemble  regression  algorithm.  The  Kalman  Filter 
used  in  a  previous  study  and  the  ensemble  regression 
algorithm  model  were  employed  across  all  sessions 
to  compute  their  respective  mean  squared  error.  The 
ensemble  regression  approach  significantly  reduced 
the  average  deviation  between  the  intended  and 
actual  finger  movements. 
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How  were  the  results  disseminated  to  communities  of  interest? 


The  development  of  the  improved  virtual  reality  environment  for  patient  training  and 
improved  decoding  algorithms  have  been  disseminated  through  presentations  at  scientific 
meetings,  invited  seminars,  radio  and  television  news  programs,  and  scientific  publications. 

What  do  you  plan  to  do  during  the  next  reporting  period  to  accomplish  the  goals? 

Obtain  HRPO  approval  and  begin  patient  recruitment.  The  anticipate  implant  first  patient  in 
the  second  quarter  of  2017. 

If  the  poly-LIFE  arrays  cannot  be  fabricated  by  the  time  the  protocol  is  approved  the  first 
patient(s)  will  be  implanted  with  the  rigid  silicon  based  electrode  arrays.  As  the  compliant 
ploy-LIFE  arrays  are  available  we  will  switch  to  implanting  them.  We  submit  and  obtain  the 
appropriate  approvals  for  utilizing  this  technology. 

All  of  the  infrastructure  for  performing  the  proposed  work  is  in  place,  so  once  protocol 
approval  is  obtained  all  of  the  goals  and  milestones  should  occurred  at  the  projected 
intervals. 

4.  IMPACT: 

What  was  the  impact  on  the  development  of  the  principal  discipline(s)  of  the  project? 

The  development  of  the  immersive  virtual  reality  environment  will  enable  new  techniques 
and  measurements  of  patient  rehabilitation.  We  will  be  able  to  have  patient  practice  with 
their  virtual  reality  prosthetic  hand  prior  electrode  implantation.  This  will  likely  increase  and 
improve  the  quality  of  the  neural  signals  present  in  the  residual  nerve  and  thus  improve  the 
control  over  the  prosthetic  hand  once  the  electrode  array  is  implanted. 

By  using  machine  learning  algorithms  that  utilize  both  time  and  frequency  domain  features 
we  have  improved  the  performance  prosthetic  control.  To  date  control  of  advanced 
prosthetic  hands  have  utilized  linear  decoding  algorithms  that  do  not  incorporate  the 
knowledge  the  most  neural  signals  encode  information  in  a  nonlinear  manner.  These 
improved  algorithms  will  likely  increase  patient  acceptance  of  advance  prosthetic  hands,  as 
even  a  few  percent  improvement  in  control  translates  into  fewer  unwanted  movements, 
e.g.  dropping  a  help  object. 

Providing  sensory  feedback  of  contact  with  grasped  objects  and/or  the  proprioceptive  sense 
of  finger  position  using  micro-stimulation  will  also  likely  increase  the  patients'  perceived 
embodiment  of  a  prosthetic  hand,  and  therefore  patients'  acceptance  and  use  of  the  hand. 

What  was  the  impact  on  other  disciplines? 

The  development  and  use  of  compliant  poly-LIFE  arrays  may  enable  multiple 
applications  that  necessitate  long-term  peripheral  nerve  interfaces:  bowel/bladder 
control  in  paralyzed  patients,  regulated  hypertension,  and  treatment  of  epilepsy.  The 
machine  learning  algorithms  we  have  developed  for  decoding  neural  signal  in  residual 
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peripheral  nerves  can  also  be  applied  to  neural  signals  recorded  from  the  central 
nervous  system  for  application  to  provide  control  for  severely  paralyzed  patients. 

What  was  the  impact  on  technology  transfer? 

An  invention  disclosure  on  the  use  and  surgical  placement  of  the  compliant  electrode 
arrays  was  filed  with  Arizona  Technology  Enterprises  (AzTE). 

What  was  the  impact  on  society  beyond  science  and  technology? 

This  research  has  prompted  several  radio  and  television  interviews  on  the  use  of  neural 
prosthetic  devices.  These  interviews  provide  the  public  with  information  about  the 
benefits  of  this  technology  and  the  ethical  issues  that  surround  its  use. 

5.  CHANGES/PROBLEMS: 

Changes  in  approach  and  reasons  for  change 

Given  the  high  failure  rate  observed,  we  are  planning  to  switch  to  a  more  compliant 
electrode  array  technology  that  has  a  better  chance  of  serving  as  platform  for  the  long¬ 
term  control  of  a  prosthetic  hand. 

Actual  or  anticipated  problems  or  delays  and  actions  or  plans  to  resolve  them 

Actual  delays: 

Regulatory  approvals  through  Mayo  Clinic  and  ASU  IRB  need  to  be  routed  through 
multiple  committees  at  both  institutions. 

Anticipated  delays: 

Patient  recruitment  is  an  unpredictable  process.  Identifying  and  recruiting  the 
appropriate  initial  patient  may  result  in  delays. 

Changes  that  had  a  significant  impact  on  expenditures 

No  to  report. 

Significant  changes  in  use  or  care  of  human  subjects,  vertebrate  animals,  biohazards,  and/or 
select  agents 

None  to  report. 

Significant  changes  in  use  or  care  of  human  subjects 

None  to  report. 

Significant  changes  in  use  or  care  of  vertebrate  animals. 

Not  applicable. 

Significant  changes  in  use  of  biohazards  and/or  select  agents 

Not  applicable. 
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6.  PRODUCTS:  List  any  products  resulting  from  the  project  during  the  reporting  period.  If  there  is 
nothing  to  report  under  a  particular  item,  state  "Nothing  to  Report. " 

Publications,  conference  papers,  and  presentations 

Subash  Padmanaban,  Tyler  Davis,  Bradley  Greger,  Decoding  of  dexterous  finger  movements 
from  neural  signals  recorded  from  human  peripheral  nerve  with  machine  learning. 

Scientific  journal  paper  written  as  part  of  Subash  Padmanaban's  PhD  thesis.  Currently  in 
preparation  for  submission. 

Inventions,  patent  applications,  and/or  licenses 

None  to  report. 

Other  Products 

None  to  report. 
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Funding  Support: 
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Name: 

Kevin  O'Neil  III 

Project  Role: 

Graduate  Student 

Researcher  Identifier 

Nearest  person  month  worked: 

1 

Contribution  to  Project: 

Kevin  O'Neil  III  has  been  developing  the  VR  prosthetic  hand  and 
immersive  environment,  and  programing  the  patient  cart 

Funding  Support: 

ASU  Dean's  Fellowship,  CDMRP 

Name: 

Subash  Padmanaban 
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Project  Role: 

Graduate  Student 

Researcher  Identifier 

Nearest  person  month  worked: 

1 

Contribution  to  Project: 

Subash  Padmanaban  has  been  developing  machine  learning 
algorithms 

Funding  Support: 

CDMRP,  TA 

Name: 

Cody  Barton 

Project  Role: 

Graduate  Student 

Researcher  Identifier 

Nearest  person  month  worked: 

3 

Contribution  to  Project: 

Cody  Barton  has  been  investigating  the  use  of  high-frequency  power 
as  an  adjunct  to  action  potential  recordings  for  controlling  prosthetic 
hands 

Funding  Support: 

CDMRP 

Name: 

Denise  Oswalt 

Project  Role: 

Graduate  Student 

Researcher  Identifier 

Nearest  person  month  worked: 

1 

Contribution  to  Project: 

Denise  Oswalt  had  been  developing  and  testing  nonlinear  support 
vector  machines  for  decoding  neural  signals 

Funding  Support: 

CDMRP,  TA 

Has  there  been  a  change  in  the  active  other  support  of  the  PD/PI(s)  or  senior/key  personnel  since  the 
last  reporting  period? 

Nothing  to  Report. 

What  other  organizations  were  involved  as  partners? 
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Mayo  Clinic  Arizona 

8.  SPECIAL  REPORTING  REQUIREMENTS 

Not  applicable. 

9.  APPENDICES: 

Davis  T,  Wark  HAC,  Hutchinson  DT,  Warren  DJ,  O'Neill  III  K,  Scheinblum  T,  Clark  GA,  Normann 
RA,  Greger  B,  Restoring  motor  control  and  sensory  feedback  in  people  with  upper  extremity 
amputations  using  arrays  of  96  microelectrodes  implanted  in  the  median  and  ulnar  nerves. 
Journal  of  Neural  Engineering  13:3  2016 

Paper  provides  information  on  earlier  study  and  support  for  proposed  changes  to  current  work. 
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CrossMark 

Abstract 

Objective.  An  important  goal  of  neuroprosthetic  research  is  to  establish  bidirectional 
communication  between  the  user  and  new  prosthetic  limbs  that  are  capable  of  controlling  >20 
different  movements.  One  strategy  for  achieving  this  goal  is  to  interface  the  prosthetic  limb 
directly  with  efferent  and  afferent  fibres  in  the  peripheral  nervous  system  using  an  array  of 
intrafascicular  microelectrodes.  This  approach  would  provide  access  to  a  large  number  of 
independent  neural  pathways  for  controlling  high  degree-of-freedom  prosthetic  limbs,  as  well  as 
evoking  multiple-complex  sensory  percepts.  Approach.  Utah  Slanted  Electrode  Arrays  (USEAs, 

96  recording/stimulating  electrodes)  were  implanted  for  30  days  into  the  median  (Subject  1-M, 

31  years  post-amputation)  or  ulnar  (Subject  2-U,  1.5  years  post-amputation)  nerves  of  two 
amputees.  Neural  activity  was  recorded  during  intended  movements  of  the  subject’s  phantom 
fingers  and  a  linear  Kalman  filter  was  used  to  decode  the  neural  data.  Microelectrode  stimulation 
of  varying  amplitudes  and  frequencies  was  delivered  via  single  or  multiple  electrodes  to 
investigate  the  number,  size  and  quality  of  sensory  percepts  that  could  be  evoked.  Device 
performance  over  time  was  assessed  by  measuring:  electrode  impedances,  signal-to-noise  ratios 
(SNRs),  stimulation  thresholds,  number  and  stability  of  evoked  percepts.  Main  results.  The 
subjects  were  able  to  proportionally,  control  individual  fingers  of  a  virtual  robotic  hand,  with  13 
different  movements  decoded  offline  (r  =  0.48)  and  two  movements  decoded  online.  Electrical 
stimulation  across  one  USEA  evoked  >80  sensory  percepts.  Varying  the  stimulation  parameters 
modulated  percept  quality.  Devices  remained  intrafascicularly  implanted  for  the  duration  of  the 
study  with  no  significant  changes  in  the  SNRs  or  percept  thresholds.  Significance.  This  study 
demonstrated  that  an  array  of  96  microelectrodes  can  be  implanted  into  the  human  peripheral 
nervous  system  for  up  to  1  month  durations.  Such  an  array  could  provide  intuitive  control  of  a 
virtual  prosthetic  hand  with  broad  sensory  feedback. 

[S]  Online  supplementary  data  available  from  stacks. iop.org/JNE/13/036001/mmedia 


5  These  authors  contributed  equally  to  this  work. 


1 741  -2560/1 6/036001  +1 5$33.00 


1 


©  2016  IOP  Publishing  Ltd  Printed  in  the  UK 


J.  Neural  Eng.  13  (2016)  036001 


T  S  Davis  ef  al 


Keywords:  neural  prosthesis,  peripheral  nerve  interface,  brain  machine  interface 
(Some  figures  may  appear  in  colour  only  in  the  online  journal) 


1.  Introduction 

The  volitional  control  of  movement  involves  a  complex  and 
integrated  hierarchy  of  sensory-motor  neural  systems.  Arrays 
of  microelectrodes  implanted  at  various  levels  in  this  hier¬ 
archy  have  been  used  to  record  spatiotemporal  patterns  of 
neural  activity  and  for  correlation  of  these  patterns  with 
sensory  stimulation  and  motor  behaviors.  Early  efforts  to 
obtain  volitional  control  signals  from  the  nervous  system 
were  conducted  in  non-human  primates  by  decoding  neural 
activity  patterns  recorded  with  microelectrodes  implanted  in 
the  motor  cortex  [1-6].  Researchers  have  been  translating 
these  efforts  to  human  subjects.  Severely  paralyzed  patients 
have  had  electrode  arrays  implanted  into  areas  of  the  cerebral 
cortex  involved  in  motor  control.  Using  these  electrode  arrays 
electrophysiological  recordings  of  action  potential  firing  from 
neurons  have  provided  control  over  external  devices  ranging 
from  two-dimensional  movements  of  computer  cursors  to  10 
degrees-of-freedom  (DOF)  robotic  arms  and  hands  [7-16]. 
Investigations  of  electrode  arrays  that  rest  on  the  surface  of 
the  cerebral  cortex,  i.e.  electrocorticography  and  micro-elec- 
trocorticography,  have  been  perform  in  patients  undergoing 
neural  surgery  for  the  treatment  of  epilepsy.  These  surface 
electrodes  record  the  aggregate  signals  from  groups  of  neu¬ 
rons,  rather  than  from  individual  neurons,  but  are  still  capable 
of  providing  control  signals  for  prosthetic  devices  [17-31].  In 
these  studies  the  subjects  relied  on  visual  or  auditory  sensory 
input  to  provide  feedback  of  their  performance,  and  ongoing 
research  is  investigating  using  central  nervous  system  sti¬ 
mulation  to  provide  feedback  from  prosthetic  limbs  [32,  33]. 
The  central  nervous  system  is  an  effective  implantation  site 
for  people  with  paralysis,  however  investigations  of  utilizing 
peripheral  nerve  approaches  such  as  targeted  reinnervation 
[34,  35]  or  peripheral  nerve  electrode  implantation  [36-45] 
are  warranted  for  people  with  amputations. 

The  current  state-of-the-art  for  people  with  upper-limb 
amputations  are  mechanically  or  myoelectrically  controlled 
prosthetic  arms  with  few  DOF  that  do  not  provide  sensory 
feedback  [46].  Next  generation  prosthetic  arms  are  being 
developed  with  upwards  of  26  DOF  [47,  48]  and  with 
embedded  sensors  intended  to  be  used  to  provide  sensory 
feedback  to  the  user  [37,  44,  49].  Several  strategies  are  being 
investigated  for  providing  sensory  feedback  from  these  new 
prosthetic  limbs  including  surface  vibrators  [49],  and  extra- 
neural  [41,  42,  45,  50]  or  intraneural  [44,  51-54]  electrodes. 
The  number  of  extra/intraneural  electrodes  (or  ‘contacts’) 
implanted  into  each  human  peripheral  nerve  has  been 
increasing,  with  previous  studies  having  used:  one  [37,  40], 
six  [55],  eight  [45,  56],  sixteen  [44],  or  twenty  [53]  electrodes 
to  established  bidirectional  communication  with  a  subject’s 
peripheral  nervous  system. 

Modern,  high-DOF  bidirectional  prosthetic  arms  and 
hands  may  require  numerous  microelectrodes  to  obtain  suf¬ 
ficiently  independent  control  and  feedback  signals.  Motor 


intent  has  been  decoded  online  and  offline  [37,  43,  44,  55] 
with  control  of  up  to  three  different  movements  from  offline 
decodes  [37,  43,  44],  and  sensory  feedback  has  been  provided 
via  electrical  stimulation  [37,  38,  40,  44,  45,  49,  56,  57]  with 
the  evocation  of  a  maximum  of  nine  different  percepts  for  a 
single  neural  implant  [44], 

We  report  herein  an  approach  for  the  control  of  prosthetic 
limbs  using  an  array  of  96  microelectrodes,  the  Utah  Slanted 
Electrode  Array  (USEA)  [52],  which  increases  the  number  of 
electrodes  that  have  been  used  to  interface  with  a  peripheral 
nerve  by  80  more  electrodes  per  nerve  over  previously 
investigated  neural  interfaces  [44,  45].  This  work  was  con¬ 
ducted  in  two  subjects  who  had  previously  undergone  trans¬ 
radial  amputations.  Each  subject  had  a  microelectrode  array 
implanted  in  one  of  their  transected  peripheral  nerves  in  order 
to  selectively  access  individual  or  small  groups  of  motor  and/ 
or  sensory  fibres.  Selective  spatiotemporal  neural  activity 
patterns  were  recorded  when  subjects  were  asked  to  make 
volitional  movements  of  their  phantom  fingers.  Closed-loop 
neural  control  of  multiple  cursors  or  virtual  robotic  fingers 
was  achieved  in  each  subject  with  two  DOF  control  achieved 
online  for  both  subjects. 

Electrophysiological  recordings  from,  and  electrical  sti¬ 
mulation  of,  peripheral  nerves  using  transversal  multichannel 
intrafascicular  electrodes  have  enable  human  subjects  to  make 
three  different  grasp  movements  utilizing  sensory  feedback 
provided  by  sensors  in  a  prosthetic  hand  [43,  44].  Up  to  eight 
sensory  percept  were  evoked  using  long-term  implanted 
peripheral  nerve  cuff  electrodes  that  did  not  penetrate  the 
nerve  [45].  We  investigated  electrophysiological  recordings 
from,  and  stimulation  of,  human  peripheral  nerves  with  high- 
count  (96  electrodes)  intrafascicular  electrodes.  Utilizing  this 
device  we  show  that  up  to  13  different  movements  can  be 
decoded  from  human  peripheral  nerve  signals  using  visual 
feedback  of  movements.  Additionally,  we  show  that  up  to  86 
percepts  (number  of  percepts  in  a  single  stimulation  session) 
with  an  average  of  8 1  percepts  over  the  study  duration  can  be 
evoked.  Together  these  studies  demonstrate  that  an  array  of 
electrodes  interfacing  with  residual  nerves  of  patients  with  a 
forearm  amputation  allow  for  selective  access  of  both  sensory 
and  motor  nerve  fibres.  These  results  further  suggest  that 
intuitive  and  dexterous  control  of  prosthetic  fingers  with 
sensory  feedback  can  be  provided  for  future  bidirectional 
prosthetic  limbs  using  an  array  of  intrafascicularly  and  peri- 
fascicularly  implanted  microelectrodes. 


2.  Methods 

This  study  was  approved  by  the  University  of  Utah  Institu¬ 
tional  Review  Board,  the  Salt  Lake  City  Veterans  Affairs 
Hospital  Research  and  Development  Service  Center  and  the 
US  Federal  Defense  Advanced  Research  Projects  Agency. 
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2. 1.  Pre-study  enrolment  period 

Two  volunteers  with  previous  transradial  amputations  parti¬ 
cipated  in  this  study  and  underwent  implantation  for  a  one 
month  period  with  a  USEA  into  their  median  nerve  (Subject 
1-M,  Median  nerve  implant,  31  years  post-amputation)  or 
ulnar  nerve  (Subject  2-U,  Ulnar  nerve  implant,  1.5  years  post¬ 
amputation).  Potential  volunteers  were  evaluated  for  the 
extent  by  which  they  perceived  that  they  were  able  to  make 
specific  movements  with  their  phantom  fingers.  The  duration, 
frequency,  and  intensity  of  phantom  limb  sensations, 
including  pain,  were  documented  in  the  patient’s  journal 
throughout  the  study.  Multiple  phantom  movements  mediated 
by  median  and  ulnar  nerve  activity  were  evaluated  in  each 
volunteer,  and  the  extent  of  their  ability  to  move  their  fingers 
was  noted  from  their  descriptions  (e.g.  Subject  1-M  had  ‘a 
very  fluid  hand’  but  Subject  2-U’s  phantom  fingers  were  often 
‘clenched  tight’).  After  selection  for  inclusion  in  the  study, 
volunteers  were  then  given  a  mirror  box  and  specific  exercises 
to  perform  in  order  to  strengthen  their  perceived  ability  to 
move  the  digits  on  their  phantom  hands.  Volunteers  were 
asked  to  document  any  changes  in  the  perceived  control  of 
their  phantom  hand,  and  if  any  decreased  control  or  unplea¬ 
sant  sensation  was  experienced,  the  volunteers  would  have 
been  asked  to  discontinue  use  of  the  mirror  box  (none  of  the 
volunteers  experienced  such  changes).  Prior  to  enrolment, 
volunteers  underwent  a  psychosocial  evaluation  in  order  to 
determine  if  any  underlying  psychological  conditions  were 
present  which  would  have  excluded  them  from  the  study. 

2.2.  Nerve  electrode  arrays 

The  USEA  (Blackrock  Microsystems,  Salt  Lake  City,  UT, 
USA)  has  been  described  elsewhere  [52]  and  will  only  be 
briefly  described  herein.  The  array  consists  of  100  electrodes 
(96  recording/ stimulating  electrodes,  4  electrodes  wired  as 
unused  backup  reference  electrode)  with  lengths  ranging  from 
0.5  to  1.5  mm  (10  by  10  grid  with  400  pm  spacing)  that 
project  out  from  a  4  x  4  x  0.3  mm3  substrate.  Lead  wire 
lengths  from  the  connector  (custom-built  item  compatible 
with  the  ZIF  Clip  96,  Tucker  Davis  Technologies,  Inc., 
Alachua,  FL,  USA)  to  the  array  were  configured  for 
implantation  into  the  upper  limb  nerves  at  a  transradial 
location.  Lead  wires  included:  (1)  100  lead  wires  (~9cm 
long)  with  96  of  the  wires  bonded  to  electrodes  used  for 
recording  and  4  bonded  to  backup  reference  electrodes,  (2) 
two  low-impedance  platinum  reference  wires  ( — 8.5  cm  long) 
connected  to  the  reference  channel,  and  (3)  two  low-impe¬ 
dance  platinum  ground  wires  (~7.5  cm  long)  connected  to  the 
recording  amplifier  ground  and  stimulation  return.  Approxi¬ 
mately  10  mm  of  the  distal  ends  of  the  reference  and  ground 
wires  were  twisted  and  looped  back,  with  distal  ends  secured 
onto  the  proximal  reference  or  ground  wires  by  coating  with 
silicone  (MED-4211,  NuSil  Technology  LLC,  Carpinteria, 
CA,  USA).  Thus,  each  loop  consisted  of  two  wires  (two 
reference  or  two  ground  wires),  and  the  distal  ends  of  these 
loops  were  de-insulated  to  reduce  their  impedance.  The 
lengths  of  the  reference  and  ground  wires  were  measured 


from  the  base  of  the  connector  to  the  end  of  the  distal  loop. 
The  reference  and  ground  wires  were  placed  inside  the  nerve 
wrap  (see  section  2.3  surgical  procedures)  near  the  electrode 
array. 

A  Neuroport  data  acquisition  system  (Blackrock  Micro¬ 
systems,  Salt  Lake  City,  UT,  USA)  was  used  to  measure  the 
electrode  impedances.  A  sinusoidal  current  at  1  kHz  was 
passed  through  a  reference  electrode,  and  impedance  was 
simultaneously  computed  on  all  electrodes.  Before  implant¬ 
ation,  the  electrodes  on  each  device  had  an  average 
(mean  ±  std)  impedance  of  75  ±  57  kD  (Subject  1-M)  and 
90  ±  28  kfl  (Subject  2-U).  Working  electrodes  were  defined 
as  electrodes  with  an  impedance  <500  kfi. 

2.3.  Surgical  procedures 

The  distal  nerve  end  was  exposed  and  the  implant  site  was 
selected  such  that  it  was  distal  to  any  branching  points  and 
proximal  to  the  transitional  zone  adjacent  to  the  neuroma 
(approximately  5-10  mm  from  the  neuroma).  The  USEA  was 
then  passed  transcutaneously  via  a  trocar  (7-8  mm  diameter) 
down  to  the  exposed  implantation  site.  A  metal  platform  was 
placed  underneath  the  nerve,  along  with  a  high  visibility 
background  and  a  reconstituted  organic  nerve  wrap  (Axo- 
Guard  Nerve  Wrap,  AxoGen  Inc.,  Alachua,  FL,  USA).  The 
lead  wires  were  then  sutured  (8-0  nylon)  to  the  epineurium 
(~5  mm  from  the  base  of  the  electrode  array).  The  USEAs 
were  inserted  with  a  pneumatic  insertion  device  [31]  that  was 
hand-held  by  the  surgeon.  The  implanted  USEA,  reference 
wires,  ground  wires,  and  nerve  were  contained  within  the 
organic  nerve  wrap,  which  was  closed  snuggly  around  the 
implant  site  with  titanium  vascular  clips  (supplementary 
figure  1).  The  organic  nerve  wrap  was  sutured  (8-0  nylon)  to 
the  epineurium  proximal  and  distal  to  the  array  site  to  prevent 
movement  of  the  wrap  along  the  nerve. 

The  percutaneous  site  was  dressed  with  a  1"  diameter, 
clorohexadine  antibacterial  patch  (Biopatch,  Ethicon  Inc., 
Johnson  &  Johnson,  New  Brunswick,  NJ,  USA).  In  Subject 
2-U,  the  connector  was  sutured  down  to  the  skin  to  prevent 
stress  on  the  lead  wires  due  to  movement  of  the  connector. 
The  entire  percutaneous  site  was  layered  with  gauze  and 
covered  with  a  breathable  and  waterproof  transparent  dressing 
(Tegaderm,  3M  Healthcare,  St.  Paul  MN,  USA).  The  gauze 
and  film  dressing  was  changed  during  each  experimental 
session  and  antibacterial  patches  were  replaced  every  7-10 
days.  To  decrease  the  inflammatory  process  and  potentially 
assist  in  enhanced  signal  quality  over  time,  subjects  were 
given  dexamethasone  (0.1  mg  kg-1  IV,  Mylan  Institutional 
LLC,  Rockford,  IL)  intraoperatively  after  removal  of  the 
tourniquet  and  minocycline  (100  mg  BID,  Watson  Pharma¬ 
ceuticals,  Parsippany,  NJ)  2  days  prior  and  for  5  days  after 
surgery  [58,  59]. 

At  the  end  of  the  one  month  experimental  period,  each 
subject  underwent  explantation  of  the  USEAs  under  general 
anaesthesia.  The  percutaneous  wires  were  cut  at  the  level  of 
the  skin  and  the  entire  limb  was  then  prepared  for  surgery. 
The  implant  side  was  exposed  and  the  lead,  reference,  and 
ground  wires  were  cut  adjacent  to  the  organic  nerve  wrap 
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containment  system  and  all  wires  were  removed.  The  entire 
implant  site  and  neuroma  was  excised.  The  new  nerve  end 
was  then  sutured  deep  in  the  surrounding  musculature 
according  to  standard  surgical  procedures  for  neuroma 
excisions. 

2.4.  Experimental  sessions 

Two  hour  experimental  sessions  were  performed  on  an 
average  of  three  times  per  week.  The  time  was  limited  by 
subject  availability  or  their  willingness  to  continue  testing 
(Subject  1-M  underwent  12  total  experimental  sessions:  6 
electrophysiological  recording  sessions  and  8  microstimula¬ 
tion  sessions;  Subject  2-U  underwent  a  total  of  14  exper¬ 
imental  sessions:  13  electrophysiological  recording  sessions 
and  8  microstimulation  sessions).  All  experimental  sessions 
were  recorded  with  a  video  camera,  which  was  time-stamped 
to  the  neural  recording  or  stimulation  data. 

2.5.  Neural  recordings,  decoding  and  instrumentation 

Neural  signals  were  amplified  and  recorded  using  active  head- 
stage  cables  (ZIF-Clip  96,  Tucker  Davis  Technologies,  Inc., 
Alachua,  FL,  USA)  that  connected  to  a  custom-built  inter¬ 
connect  board  used  to  interface  the  ZIF-Clip  cable  with  the 
Neuroport  data  acquisition  system.  The  continuous  neural 
signals  were  band-pass  filtered  with  cutoff  frequencies  of 
0.3  Hz  (lst-order  high-pass  Butterworth  filter)  and  7500  Hz 
(3rd-order  low-pass  Butterworth  filter)  and  sampled  at 
30  kHz.  Online  multi-unit  activity  was  extracted  from  high- 
pass  filtered  recording  data  (250  Hz  4th-order  Butterworth 
filter)  by  setting  a  threshold  using  the  auto  threshold  setting  in 
the  Neuroport  data  acquisition  software  (multiplier  =  3, 
threshold  =  multiplier  x  noise  estimate  of  the  signal).  For 
each  of  the  96  neural  recording  channels,  multi-unit  neural 
firing  rates  were  calculated  using  unsorted  spikes  and  a 
moving  box-car  average  of  300  ms  with  an  update  period  of 
33  ms.  Offline,  action  potentials  were  isolated  from  the  high- 
pass  filtered  data  using  commercially  available  software 
(Offline  Sorter  version  3,  Plexon  Inc.,  Dallas,  Texas,  USA). 
Signal-to-noise  ratios  (SNRs)  were  calculated  by  dividing  the 
mean  peak-to-peak  action  potential  amplitude  by  two  times 
the  standard  deviation  of  the  recorded  noise  [60]. 

2.5.1.  Decoding  neural  signals  and  control  of  virtual  robotic 
fingers  or  computer  indicators.  A  standard  Kalman  filter  was 
implemented  to  perform  the  continuous  neural  decodes  [61]. 
This  algorithm  assumes  a  linear  relationship  between  the 
kinematics  (finger  position)  and  the  neural  data.  For  this 
study,  it  was  used  to  provide  continuous  estimates  of  finger 
position  based  on  the  firing  rates  of  multiple  neurons. 
Sessions  began  by  cueing  the  subjects  to  perform  multiple 
flexions,  extensions  or  abductions  of  their  individual  phantom 
fingers.  A  total  of  4  movements  were  performed  for  Subject 
1-M  (Thumb-Flex,  Index-Flex,  Middle-Flex,  and  Ring-Flex) 
and  13  movements  for  Subject  2-U  (Thumb-Flex,  Thumb- 
Extend,  Index-Flex,  Index-Extend,  Index-Abduct,  Middle- 
Flex,  Middle-Extend,  Ring-Flex,  Ring-Extend,  Ring-Abduct, 


Little-Flex,  Little-Extend,  Little- Abduct).  These  movements 
are  given  acronyms  based  on  the  first  one  or  two  letters  of 
each  word,  which  are  used  in  subsequent  figures  (e.g., 
Thumb-Flex  =  TF).  The  subjects  were  instructed  to  make 
finger  movements  by  either  a  computer  controlled  display  of 
indicators  (Subject  1-M)  or  by  movement  of  virtual  robotic 
fingers  [62]  (Subject  2-U)  (supplementary  figure  1).  Subject 
1-M  held  a  small  manipulandum  consisting  of  individual, 
movable  pads  that  could  be  depressed  by  each  finger  with  the 
subject’s  intact  hand.  For  Subject  1-M  finger  position  targets 
were  provided  on  a  computer  monitor.  In  order  to  have  a 
metric  of  the  finger  positions  of  the  phantom  hand,  Subject 
1-M  was  asked  to  mirror  the  movements  made  with  their 
phantom  fingers  by  pressing  on  the  pads  of  a  manipulandum 
with  their  intact  fingers.  During  algorithm  training  for  Subject 
1-M  the  finger  position  of  the  phantom  hand  (instruction 
variable)  was  measured  as  the  continuous  voltage  signal  from 
pressure  sensors  on  the  pads  of  the  manipulandum,  which  was 
displayed  by  a  graphical  indicator  on  the  computer  monitor. 
For  Subject  2-U  the  instructions  on  what  finger  movements  to 
make  were  provided  using  the  virtual  prosthetic  hand.  During 
algorithm  training  the  virtual  prosthetic  hand  made  specific 
finger  movements  under  computer  control  and  Subject  2-U 
was  asked  to  mimic  these  movements  with  their  phantom 
hand.  The  computer  generated  positions  of  the  virtual 
prosthetic  fingers  (instruction  variable)  was  generated  using 
a  cosine  function,  which  was  normalized  from  —1  (full 
extension/adduction)  to  +1  (full  flexion/abduction).  Multi¬ 
unit  firing  rates  from  selected  electrodes  and  the  movement 
instruction  variables  were  then  used  to  train  and  test  the 
decode  algorithm  for  both  online  and  offline  control. 

2.5.2.  Online  neural  decodes.  For  online  decoding, 
electrodes  were  selected  based  on  their  ability  to  record 
movement-correlated  action  potentials.  This  selection  process 
was  made  using  two  methods:  (1)  the  experimenters  viewed  a 
map  of  correlation  coefficients  between  the  instruction 
variables  and  the  firing  rates  on  each  electrode  and  (2)  the 
experimenters’  subjective  observations  of  the  high-pass 
filtered  neural  data  on  each  electrode  during  the  cued 
movements.  Following  electrode  selection,  the  decode 
algorithm  was  trained  on  a  set  of  10  trials  for  each 
movement  type.  For  testing,  subjects  were  asked  to  control 
a  computer  display  of  indicators  (Subject  1-M)  or  virtual 
prosthetic  fingers  (Subject  2-U)  and  acquire  targets  in  a  trial- 
based  format. 

Subjects  began  a  trial  by  moving  the  fingers  of  their 
phantom  hand  to  a  neutral,  hand-at-rest  position,  which 
moved  those  indicators  or  virtual  prosthetic  fingers  under 
neural  control  to  the  hand-at-rest  start  position.  The  indicators 
and  virtual  prosthetic  fingers  not  under  neural  control  were  set 
to,  and  held  at,  the  start  position  by  the  computer.  After  a  few 
hundred  milliseconds,  one  or  more  of  the  computer  generated 
the  targets  would  change  spatial  location  and  the  subjects 
were  required  to  match  finger  positions  of  all  prosthetic 
fingers  under  neural  control  to  the  new  positions  of  the 
computer  generated  targets  for  at  least  300  ms  and  up  to 
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3000  ms.  Targets  were  presented  of  varying  diameters  (30%- 
36%  of  full-range  movement)  and  distances  (40%-100%  of 
full-range  movement)  from  the  starting  point.  In  order  to 
correctly  complete  a  trial,  the  subjects  had  to  acquire  and 
maintain  the  tip(s)  of  the  finger(s)  of  the  virtual  prosthetic 
hand  under  neural  control  at  the  target(s).  At  the  completion 
of  a  trial  the  target(s)  would  return  to  the  hand-at-rest  start 
position  and  the  subjects  would  have  to  return  the  indicators 
or  fingers  of  the  virtual  prosthetic  hand  to  the  start  position  in 
order  to  begin  a  new  trial.  Throughout  the  trials  subjects  were 
instructed  to  maintain  their  phantom  fingers  that  were  not 
being  instructed  to  move,  i.e.  corresponding  to  the  fingers  on 
the  virtual  prosthetic  hand  that  were  not  under  neural  control, 
at  their  start  positions.  This  task  paradigm  assessed  the  ability 
of  subjects  to  move  the  virtual  indicator/ fingers  indepen¬ 
dently  and  proportionally.  The  finger  required  to  move  to  the 
target  and  the  position  of  the  target  would  be  changed  for 
different  trials.  This  produced  individuated  proportional 
finger  movements  during  simultaneous  neural  control  of 
multiple  fingers  (see  supplemental  video  2). 

2.5.3.  Offline  neural  decodes.  For  offline  decoding,  an 
average  of  22  (range  of  13-37)  trials  of  each  movement 
type  recorded  during  a  typical  experimental  session  were 
analyzed.  Electrodes  were  chosen  on  the  basis  of  the  results  of 
a  Wilcoxon  signed-rank  test.  First,  a  ‘baseline-period’  was 
defined  as  the  2  s  period  prior  to  the  onset  of  the  movement 
cue,  and  a  ‘movement-period’  was  defined  as  the  2  s  period 
after  the  onset  of  the  cue.  Then,  the  difference  in  median 
firing  rates  was  calculated  between  the  ‘movement-period’ 
and  the  ‘baseline-period’  for  all  movement  types  and  trials 
recorded  on  each  electrode.  The  null  hypothesis  was  that  the 
data  came  from  a  continuous,  symmetric  distribution  with  a 
median  difference  equal  to  zero  (i.e.,  the  electrode  did  not 
record  increased  firing  rates  in  the  movement-period 
compared  with  the  baseline-period).  All  electrodes  for 
which  the  null  hypothesis  was  rejected  (p  <  0.001)  with  a 
positive  median  difference  from  baseline  were  kept.  These 
electrodes  were  then  sorted  in  order  of  increasing  median 
difference  and  the  top  90%  of  electrodes  were  used  in  the 
offline  decode.  After  electrode  selection,  the  decode  algorithm 
was  trained  on  the  first  10  trials  for  each  movement  and  tested 
on  the  remaining  3-27  trials.  A  Pearson’s  correlation 
coefficient  ( r )  between  the  instruction  variable  and  the 
Kalman  filter  estimate  of  finger  positions,  calculated  for  the 
entire  continuous  sampling  of  data  corresponding  with  the 
trained  movements  for  the  remaining  3-27  testing  trials,  was 
used  to  quantify  the  decode  performance. 

2.6.  Neural  stimulation  and  instrumentation 

Current-controlled,  biphasic,  cathodic-first  (without  anodic 
bias)  stimulation  [63]  (IZ2,  128-channel  Tucker-Davis 
Technologies  Stimulator,  Inc.,  Alachua,  FL,  USA)  was 
delivered  to  individual  or  subsets  of  electrodes  using  custom 
Lab  VIEW  software  (National  Instruments  Corp.,  Austin,  TX, 
USA),  using  the  platinum  ground  wires  as  a  return.  The 


stimulator  had  a  compliance  voltage  of  ±15  V  (LZ48-400, 
Tucker-Davis  Technologies  Stimulator,  Inc.,  Alachua,  FL, 
USA).  Maximum  stimulation  was  limited  by  either:  (1)  the 
comfort  level  of  the  subject,  (2)  a  perceived  change  in  the 
quality  of  the  percept  or  (3)  if  the  maximum  safety  limit  for 
delivering  electricity  to  tissue  was  reached  [64].  The  safety 
limit  for  injecting  charge  into  the  tissue  was  determined  by 
measuring  the  maximum  cathodic  voltage  excursion  (with  a 
safety  limit  of  —0.6  V)  across  the  electrode  and  ground  during 
stimulation  [65-67].  The  voltage  drop  due  to  tissue  impe¬ 
dance  was  not  subtracted  in  our  calculations,  resulting  in  a 
conservative  estimate  of  safe  stimulation  parameters.  Stimu¬ 
lation  parameters  varied  depending  on  the  objective  of  the 
experimental  session.  Pulse  amplitude,  frequency,  and  train 
duration  ranged  from  1  to  100  p A,  1  to  320  Hz,  and  0.2  to 
60  s,  respectively.  Pulse  width  and  inter-phase  interval  were 
held  fixed  at  200  /is  and  100  ps. 

2.6. 1.  Stimulation  across  different  amplitudes  and  frequencies 
( Subject  1-M).  In  Subject  1-M,  stimulation  experiments  were 
focused  on  a  subset  of  electrodes  to  investigate  the  subject’s 
ability  to  detect  and  discriminate  single  and  multiple  sensory 
percepts  and  the  effects  of  modulating  stimulation  frequency 
on  sensory  percepts.  A  custom-built  software  interface 
(LabVIEW  software,  National  Instruments  Corp.,  Austin, 
TX,  USA)  was  used  to  allow  Subject  1-M  to  control  the 
amplitude  of  stimulation  (1-100  pA  in  steps  of  1  //A)  at  a 
constant  frequency  (200  Hz)  and  train  duration  (0.2  s)  in  order 
to  determine  the  threshold  of  a  sensory  percept  (sessions  1-8, 
post-implantation  days  5-26).  Control  of  frequency 
( 1-320  Hz  in  1  Hz  steps,  threshold  amplitudes,  0-60  s 
durations)  was  provided  to  the  subject  by  pressing  down  on 
a  pressure  sensor  mounted  on  the  manipulandum  with  a  finger 
from  the  subject’s  intact  hand  (sessions  4-8,  post¬ 
implantation  days  14-26). 


2.6.2.  Simultaneous  multielectrode  stimulation  ( Subject 
1-M).  Stimulation  was  delivered  via  multiple  electrodes 
simultaneously  to  investigate  if  the  subject  could  discern 
multiple  electrically  evoked  percepts  simultaneously. 
Stimulation  sessions  were  performed  where  the  subject  was 
cued  that  a  trial  began,  but  did  not  know  whether  stimulation 
was  delivered  via  either  of  two  electrodes,  both  electrodes 
simultaneously,  or  no-stimulation  was  delivered.  Such 
blinded-trial  data  was  collected  during  three  different 
experimental  sessions  on  post-implantation  days  13,  19,  and 
26.  Electrodes  with  different  inter-electrode  distances  were 
chosen,  and  supra-threshold  stimulation  amplitudes  were  used 
(day  13 — electrodes  16  and  19,  1200  pm  distance,  19  pA  and 
25  pA  thresholds;  day  19 — electrodes  19  and  20,  400  pm 
distance,  47  pA  and  18  pA  thresholds;  day  26 — electrodes  20 
and  46,  1442  pm  distance,  18  pA  and  10  pA  thresholds).  For 
each  trial,  the  subject  had  to  record  the  size,  location  and 
intensity  of  the  evoked  percept  on  anterior  and/or  posterior 
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maps  of  a  hand.  If  no  percept  was  evoked,  the  subject 
reported  no  sensation. 

2.6.3.  Stimulation  thresholds  for  all  96  electrodes  over  30  days 
( Subject  2-U ).  All  96  USEA  microelectrodes  were 
individually  stimulated  (six  sessions  at  200  Hz  on  post¬ 
implantation  days  6-25;  two  sessions  at  20  Hz  post¬ 
implantation  days  26-27)  and  the  threshold,  location,  size, 
quality  and  intensity  of  each  evoked  percept  was  mapped 
using  custom  built  software  (LabVIEW  software,  National 
Instruments  Corp.,  Austin,  TX,  USA).  Subject  2-U  controlled 
when  stimulation  was  delivered,  but  the  experimenter 
controlled  the  amplitude  (1-100  pA  in  steps  of  1-5  //A)  and 
pulse  train  duration  (0.2  or  2  s).  The  quality  of  the  percept  was 
designated  from  a  list  (tingle,  pressure,  vibration,  hot,  cold)  or 
could  be  defined  using  their  own  words.  When  a  percept  was 
faint  or  the  subject  was  unsure  of  the  sensation,  stimulation 
could  be  repeated  until  they  were  sure  the  percept  was 
electrically  evoked  (as  opposed  to  a  phantom  limb  sensation). 

2.6.4.  Stability  of  percept  location  over  30  days  ( Subject 
2-U).  All  of  the  electrodes  that  evoked  a  percept  during  each 
stimulation  session  when  each  electrode  in  the  entire  array 
was  stimulated  at  200  Hz  were  analyzed  (post-implantation 
days  10-25;  Subject  2-U).  A  percept  was  considered  stable  if 
it  remained  in  a  localized  anatomical  region  of  the  phantom 
hand,  defined  as  one  of  the  following:  anterior  or  posterior  of 
a  particular  finger,  the  palm,  or  the  back  of  the  hand.  If  a 
portion  of  an  evoked  percept  was  located  on  the  border 
separating  two  anatomical  locations  (e.g.,  over  the 
metacarpophalangeal  joint),  the  percept  was  considered 
within  the  location  where  the  majority  of  the  sensation 
occurred;  however,  if  the  size  of  the  percept  spread  beyond 
the  border  then  the  percept  was  not  considered  stable.  The 
first  stimulation  session  (post-implantation  day  6;  Subject 
2-U)  resulted  in  an  incomplete  mapping  of  the  array  where 
maximum  stimulation  was  not  delivered  to  all  96  electrodes. 
The  last  two  sessions  (post-implantation  days  26  and  27; 
Subject  2-U)  were  mapped  using  a  different  frequency 
(20  Hz).  These  three  stimulation  sessions  were  not  included 
in  the  spatial  stability  analysis. 

3.  Results 

The  results  presented  here  demonstrate  that,  using  a  96 
microelectrode  array  implanted  into  human  peripheral  nerves, 
up  to  13  different  movements  could  be  decoded  and  a  max¬ 
imum  of  86  percepts  could  be  evoked. 

3. 1.  Electrode  impedances 

For  Subject  1-M,  the  mean  in  vivo  impedance  for  working 
electrodes  over  the  29  day  implant  was  222  ±  133  kfl 
(mean  ±  std,  n  =  7  sessions;  figure  1(a)).  Due  to  mechanical 
failure  of  the  lead  wires,  the  number  of  working  electrodes 
dropped  from  93  to  20  by  the  end  of  week  2,  and  then  to  4  by 
the  end  of  week  4.  In  Subject  2-U,  the  percutaneous  connector 


was  sutured  to  the  skin  to  minimize  strain  applied  to  the 
wires,  and  no  systematic  failures  indicating  wire  breakages 
occurred.  The  mean  impedances  for  the  working  electrodes 
over  the  31  day  implant  for  this  subject  was  143  ±  76  kO 
(mean  ±  std,  n  =  13  sessions;  figure  1(b)),  and  the  number 
of  working  electrodes  was  87  ±  5  (mean  ±  std).  Impedances 
were  found  to  vary  significantly  over  the  course  of  the  study 
(p  <  0.0001,  Kruskal-Wallis  test,  Subject  2-U),  with  an 
increase  from  post-implantation  days  3-10. 

3.2.  Recording  and  decoding  of  motor  intent  with  high-count 
electrode  arrays 

Neural  recordings  were  made  while  the  subjects  were  asked  to 
make  volitional  flexion,  extension  or  abduction  movements  of 
their  phantom  fingers  throughout  the  study  period.  Examples 
of  action  potentials  recorded  from  the  same  electrodes  over 
time  are  shown  in  figures  1(c)  and  (d)  for  each  subject.  In 
Subject  1-M,  an  average  of  7  ±  9  electrodes  recorded  action 
potentials  for  the  6  recording  sessions  with  a  maximum  of  20 
electrodes  recording  action  potentials  on  post-implantation 
day  3  (figure  1(e)).  In  Subject  2-U,  an  average  of  22  ±  7 
electrodes  recorded  action  potentials  for  the  13  recording 
sessions  with  a  maximum  of  40  electrodes  recording  action 
potentials  on  post- implantation  day  7  (figure  1(f)).  The  mean 
SNR  of  the  action  potentials  recorded  during  all  sessions  for 
each  subject  were  5.2  ±  2.3  for  Subject  1-M  and  5.1  ±  2.8 
for  Subject  2-U  (figures  1(e)  and  (f)).  For  Subject  2-U,  after 
post-implantation  day  10,  there  was  no  significant  change  in 
the  SNR  of  the  action  potentials  for  the  remainder  of  the  study 
(p  =  0.24,  Kruskal-Wallis  test). 

3.2.1.  Recording  neural  activity  during  different  intended 
phantom  finger  movements.  Subjects  were  cued  to  make 
different  finger  movements  either  by  a  computer  cursor  or  by 
movement  of  virtual  robotic  fingers.  The  intent  to  flex  or 
extend  individual  phantom  digits  produced  spatiotemporal 
neural  firing  patterns  that  were  visible  in  the  high-pass  filtered 
recording  data  (figures  2(a)  and  (b)).  The  patterns  of  neural 
activity  varied  across  electrodes  and  movement  types.  Some 
electrodes  recorded  action  potentials  that  were  correlated  with 
a  single  movement  type  (figure  2(a),  middle  finger  flexion 
(MF),  electrode  44,  Subject  1-M)  or  movement  of  a  single 
digit  (figure  2(b),  thumb,  electrode  88,  Subject  2-U); 
however,  other  electrodes  recorded  action  potentials  that 
correlated  with  multiple  different  fingers  and  movement  types 
(figure  2(b),  electrodes  13  and  44). 

3.2.2.  Multiple  movements  could  be  decoded  offline. 
Estimated  finger  positions  showed  high  correlations 
[r  =  0.9)  with  the  instruction  variables  for  the  best  two 
movements  (IF  and  MF)  achieved  in  Subject  1-M  and  the  best 
four  movements  (TF,  lAb,  IE,  TE)  achieved  in  Subject  2-U 
(figures  2(c)  and  (d)).  Prediction  accuracy  decreased  as  the 
number  of  movements  being  estimated  increased,  with  a 
value  of  r  =  0.48  at  13  different  finger  movements  for 
Subject  2-U  (figure  2(f)).  To  validate  the  decode  results,  the 
electrode  order  was  randomly  shuffled  between  the  training 
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Figure  1.  Electrode  impedances  and  action  potential  recording  quality.  The  left  column  of  panels  (a),  (c),  and  (e)  show  data  from  Subject 
1-M;  and  the  right  column  of  panels  (b),  (d),  and  (f)  show  data  from  Subject  2-U.  (a)  and  (b)  Box-and-whisker  plots  of  impedances  taken  pre¬ 
implantation  in  saline  and  post-implantation  throughout  the  study.  The  number  below  each  boxplot  is  the  number  of  working  electrodes,  (c) 
and  (d)  Action  potentials  recorded  on  example  electrodes  at  two  time  points,  one  early  and  one  late  in  the  study,  from  each  subject,  (e)  and  (f) 
Box-and-whisker  plots  of  signal-to-noise  for  all  action  potentials  recorded  on  all  electrodes  throughout  the  study.  The  number  below  each 
boxplot  is  the  number  of  electrodes  that  recorded  action  potentials  during  the  individual  experimental  session. 


and  testing  sets,  and  the  firing  rate  data  was  decoded  again. 
Using  this  shuffled  data,  the  prediction  accuracy  dropped  to 
r  =  0. 14  for  13  different  movements  (Subject  2-U).  For 
Subject  2-U,  the  median  decode  performance  for  seven 
sessions  spanning  23  days  for  the  same  eight  movements  was 
r  =  0.62  ±  0.07  (the  5  finger  extension  movements  were 
added  on  post-implantation  day  13  and  were  not  included  in 
this  analysis). 

3.2.3.  Multiple  movements  could  be  decoded  online.  Data 
recorded  from  the  median  nerve  was  decoded  online  for 
middle  and  index  phantom  finger  flexion,  and  the  subject  was 
able  to  move  the  graphical  indicator  to  targets  specific  to  each 
finger  individually  with  a  median  time  to  trial  completion  of 
9  ±  6  s  (16  trials,  5  electrodes;  figure  2(g)).  Data  recorded 
from  the  ulnar  nerve  was  decoded  online  for  thumb  and  little 


phantom  finger  flexions,  and  the  subject  was  able  to 
independently  and  proportionally  control  each  of  the  virtual 
robotic  fingers  with  a  median  time  to  trial  completion  of 
2  ±  5  s  (79  trials,  4  electrodes;  figure  2(h))  (supplemental 
video  2).  In  each  subject,  online  decodes  were  successful  in 
providing  simultaneous  control  of  two  graphical  indicators 
(Subject  1-M)  or  two  virtual  prosthetic  fingers  (Subject  2-U) 
from  the  neural  signals  generate  by  movement  of  their 
phantom  fingers.  However,  some  crosstalk  existed  between 
these  DOF,  making  independent  control  more  difficult. 

3.3.  Intrafascicular  microstimulation  evoked  sensory  percepts 

In  Subject  1-M,  single  electrode  stimulation  on  17  electrodes 
over  the  course  of  the  study  evoked  sensory  percepts  with  an 
average  threshold  of  27  ±  20  p A  (mean  ±  std)  (figure  3(a)) 
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Figure  2.  Decoding  volitional  phantom  finger  movements  from  peripheral  nerve  action  potentials.  The  left  column  of  panels  (a),  (c),  (e),  and 
(g)  show  data  from  Subject  1-M;  and  the  right  column  of  panels  (b),  (d),  (f),  and  (h)  show  data  from  Subject  2-U.  (a)  and  (b)  High-pass 
filtered  neural  recordings  (solid  blue)  made  by  three  electrodes  during  phantom  finger  movements  for  Subject  1-M  (post-implantation  day  3) 
and  Subject  2-U  (post-implantation  day  24).  Superimposed  is  the  instructed  finger  movement  (dashed  black).  Action  potentials  can  be  seen  in 
the  high-pass  filtered  data  extending  out  of  the  noise  during  the  instmction  period,  (c)  and  (d)  Kalman  filter  predictions  for  the  best  two 
movements  for  Subject  1-M  and  the  best  four  movements  for  Subject  2-U.  Only  a  subset  of  trials  is  shown.  Prediction  correlations  of  0.9 
were  achieved  using  multi-unit  firing  rates  calculated  from  unsorted  spike  events  on  18  electrodes  (Subject  1-M,  post-implantation  day  3)  and 
55  electrodes  (Subject  2-U,  post-implantation  day  24).  (e)  and  (f)  Box-and-whisker  plots  of  offline  Kalman  filter  performance  for  all 
combinations  of  available  movements  using  the  same  electrodes  as  in  (c)  and  (d).  (g)  and  (h)  Examples  of  Kalman  filter  performance  during 
online  decode  sessions  for  each  subject.  Boxes  represent  trial-by-trial  target  locations  and  the  traces  represent  the  Kalman  filter  predictions  for 
the  intended  finger  movement.  To  successfully  complete  a  trial,  the  predicted  finger  movement  must  enter  and  remain  in  the  target  box  for  a 
specified  hold  duration  (3000  (g)  and  300  (h)  ms).  Asterisks  highlight  verbally-stated  volitional  movements  made  by  the  subject  in  the 
absence  of  a  target. 
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Figure  3.  Electrical  stimulation  can  evoke  spatially  distinct  and  stable  sensory  percepts.  The  left  column  of  panels  (a),  (c)  and  (e)  show  data 
from  Subject  1-M;  and  the  right  panels  (b),  (d),  and  (f)  show  data  from  Subject  2-U.  (a)  and  (b)  Sensory  percept  thresholds  over  the  duration 
of  the  experimental  period  for  both  subjects.  For  subject  1-M,  only  a  subset  of  electrodes  was  stimulated  each  session.  For  subject  2-U,  except 
on  post-implantation  day  6,  all  96  electrodes  were  stimulated  each  session.  The  number  below  each  boxplot  is  the  number  of  electrodes  that 
evoked  a  sensory  percept,  (c)  and  (d)  Results  from  intrafascicular  stimulation  of  human  median  and  ulnar  nerves  show  evoked  sensations  in 
the  phantom  hand  that  approximately  followed  the  expected  spatial  distributions  for  each  nerve.  The  cumulative  data  from  all 
microstimulation  sessions  is  shown,  (e)  Single  electrode  stimulation  (electrode  19  at  47  pA\  electrode  20  at  15  pA)  produced  discrete  sensory 
percepts,  and  when  the  same  electrodes  were  stimulated  simultaneously  the  two  discrete  percepts  could  be  perceived  simultaneously  (Subject 
1-M).  The  inter-electrode  distance  in  this  example  was  400  pm.  (f)  Examples  of  percepts  that  were  evoked  by  two  different  electrodes  with 
one  percept  that  was  considered  stable  across  all  five  sessions  (electrode  63;  top  row  of  hands)  and  another  that  was  stable  for  two 
consecutive  sessions  (electrode  88;  bottom  row  of  hands).  Marked  locations  were  felt  simultaneously  (Subject  2-U). 


that  were  approximately  located  in  an  expected  median  nerve 
distribution  (figure  3(c)).  In  Subject  2-U,  when  all  96  elec¬ 
trodes  were  stimulated  individually,  an  average  of  81  elec¬ 
trodes  evoked  sensory  percepts  with  a  mean  threshold  of 
12  ±  11  //A  (mean  ±  std)  (figure  3(b))  that  were  approxi¬ 
mately  located  in  an  expected  ulnar  nerve  distribution 
(figure  3(d)).  For  this  subject,  no  significant  changes  in  the 
thresholds  occurred  after  post-implantation  day  10  (p  =  0.46, 
Kruskal-Wallis  test).  Further,  a  comparison  of  all  96  elec¬ 
trodes  demonstrated  that  the  mean  percept  threshold  for  an 


electrode  over  the  duration  of  the  study  (8  stimulation  ses¬ 
sions)  was  negatively  correlated  (r  =  —0.46,  p  <  0.0001) 
with  the  number  of  days  that  an  electrode  recorded  an  action 
potential  (13  recording  sessions)  (figures  4(a)  and  (b)). 


3.3.1.  Detection  of  multiple  percepts  simultaneously  ( Subject 
1-M).  For  Subject  1-M,  blind  trial  data  was  collected  during 
three  different  experimental  sessions  on  post-implantation 
days  13,  19,  and  26  that  included  stimulation  delivered  via 
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Figure  4.  Spatial  distributions  of  percept  thresholds  and  action 
potential  recordings  for  all  96  electrodes  over  30  days,  (a)  Mean 
threshold  needed  to  evoke  a  sensory  percept  via  stimulation  on  each 
individual  electrode  over  the  duration  of  the  study  (8  total 
stimulation  sessions,  last  stimulation  session  on  day  27  post¬ 
implantation).  (b)  Number  of  days  that  each  electrode  recorded  at  an 
action  potential  over  the  duration  of  the  study  (13  total  recording 
sessions,  last  session  on  day  27  post-implantation). 


to  self-modulate  the  stimulation  frequency  from  1  to  100  Hz 
by  pushing  down  on  the  manipulandum  pressure  sensor  with 
the  subject’s  intact  hand  with  stimulation  being  delivered  to  a 
single  electrode  at  30  pA  (post-implantation  day  14).  High 
frequencies  (100-320  Hz)  evoked  an  ‘electric  shock’  like 
quality,  and  lower  frequencies  (1-25  Hz)  and  longer 
stimulation  times  (up  to  60  s)  could  evoke  more 
physiological  percepts  (e.g.,  pressure). 

3.3.3.  Location  stability  of  percepts  for  all  96  electrodes  over 
time  ( Subject  2-U).  Five  of  the  total  eight  stimulation 
sessions  (post-implantation  days  10-25)  resulted  in  a 
complete  mapping  of  all  96  electrodes  at  fixed  parameters 
of  200  Hz,  0.2  ms  durations,  200  ps  pulse  widths,  and 
amplitudes  that  varied  from  1  to  100  //  A.  Figure  3(f)  shows 
examples  of  percepts  evoked  by  two  different  electrodes  over 
these  five  sessions.  One  of  the  electrodes  (electrode  63) 
evoked  a  stable  percept  across  all  five  sessions.  The  other 
electrode  (electrode  88)  was  stable  for  two  consecutive 
sessions.  A  total  of  61  electrodes  evoked  percepts  across  all 
five  sessions.  Out  of  these  61  electrodes,  18  electrodes 
produced  percepts  that  were  considered  stable  within  a 
defined  region  (see  methods  2.6)  of  the  phantom  hand  for  two 
consecutive  sessions  (separated  by  1-3  days).  A  total  of  eight 
electrodes  evoked  stable  percepts  for  three  consecutive 
sessions,  four  electrodes  evoked  stable  percepts  for  four 
consecutive  sessions,  and  three  electrodes  evoked  stable 
percepts  across  all  five  stimulation  sessions. 


one  electrode,  a  different  electrode,  both  electrodes,  or  no 
electrodes.  The  following  electrode-pair  combinations  were 
chosen  to  include  varying  inter-electrode  distances  using 
supra-threshold  stimulation  amplitudes:  day  13 — electrodes 
16(19  //A)  and  19  (25  pA)  with  a  1200  pm  distance;  day  19 
— electrodes  19  (47  pA)  and  20  (18  pA)  at  400  pm  distance; 
day  26 — electrodes  20  (18  pA)  and  46  (10  pA)  at  1442  pm 
distance.  Subject  1-M  correctly  discriminated  between  any- 
stimulation  (stimulation  on  either  electrode  individually  or 
both  electrodes  simultaneously)  versus  no-stimulation  with 
98%  accuracy  (58/59  total  trials;  day  13 — 19/19  trials;  day 
19—20/20  trails;  day  26—19/20  trials).  Subject  1-M 
correctly  discriminated  between  spatially  distinct  percepts 
evoked  by  microstimulation  delivered  via  one  of  two 
electrodes  (n  =  5  for  each  electrode  per  session)  with  87% 
accuracy  (26/30  total  trials;  day  13 — 9/10  trials;  day  19 — 8/ 
10  trials;  day  26 — 9/10  trials).  Spatial  discrimination  was 
accurately  reported  with  electrodes  separated  by  400  pm  (day 
19)  (figure  3(e)).  Subject  1-M  was  also  able  to  discriminate 
between  simple  patterns  of  stimulation,  i.e.,  stimulation  via 
either  electrode  individually  versus  simultaneous  stimulation 
via  both  electrodes,  with  84%  accuracy  (38/45  total  trials; 
day  13 — 11/14  trials;  day  19 — 13/15  trials;  and  day  26 — 
14/16  trials). 

3.3.2.  Frequency  and  duration  of  stimulation  modulated  the 
quality  of  percepts  ( Subject  1-M).  Subject  1-M  was  allowed 


3.3.4.  Number  and  quality  of  percepts  for  all  96  electrodes 
{Subject  2-U).  The  quality  of  evoked  percepts  in  Subject  2-U 
combined  across  all  five  complete  mapping  sessions  at 
200  Hz  included:  76  ‘tingle’  percepts;  7  ‘pressure’  percepts; 
and  216  ‘vibration’  percepts.  For  the  last  two  stimulation 
sessions,  separated  by  one  day,  where  the  array  was  mapped 
at  20  Hz,  a  total  of  76  electrodes  evoked  percepts  on  both 
days,  with  17  electrodes  evoking  stable  percepts.  The  percept 
quality  evoked  during  these  sessions  included:  21  ‘tingle’ 
percepts;  19  ‘pressure’  or  ‘hairbrush’  percepts;  17  ‘vibration’ 
percepts;  and  96  ‘cold’  or  ‘air  brush’  percepts.  The  subject 
noted  that  the  ‘tingle’  and  ‘vibration’  percepts  evoked  during 
each  session  were  of  a  ‘painful’  quality. 

3.4.  Phantom  limb  sensations  occurred  post-stimulation  in  both 
subjects 

Both  subjects  differentiated  between  phantom  limb  sensations 
(the  normal  feelings  from  their  phantom  hand)  and  phantom 
limb  pains  (percepts  that  were  considered  uncomfortable). 
Subjects  reported  an  increase  in  the  occurrence  of  phantom 
limb  sensations  that  took  on  the  characteristics  of  the  sensory 
percepts  evoked  by  electrical  stimulation.  At  times,  the  post¬ 
stimulation  sensations  included  percepts  such  as  ‘pressure’  or 
‘hair  brushing  on  the  skin’.  At  other  times,  the  subjects 
reported  an  increased  occurrence  of  post-stimulation  evoked 
phantom  limb  sensations  that  were  painful,  and  the  quality  of 
such  percepts  included:  ‘electric  shock,’  ‘stinging,’  or  ‘tin¬ 
gling’.  For  each  subject,  percepts  (duration  ss  1-2  s)  began 
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after  the  first  day  of  stimulation,  and  had  peak  occurrences  of 
10  h“'  for  subject  1-M  and  2-9  IT1  for  Subject  2-U.  By  30 
days  post-explantation  of  the  electrode  array,  both  subjects  no 
longer  reported  phantom  sensations  that  were  of  the  quality  of 
stimulation-evoked  percepts. 


4.  Discussion 

In  2003,  a  human  volunteer  had  a  Utah  microelectrode  array 
(with  all  electrodes  of  the  same  length)  implanted  in  his 
median  nerve  for  96  days.  Although  this  study  provided  new 
data  for  the  field  of  peripheral  nerve  interfaces,  it  also  had 
several  limitations  including  the  following:  stimulation  and 
recording  were  carried  out  through  20  electrodes  (out  of  the 
100  electrodes  that  were  implanted);  no  action  potential  data 
were  presented  to  support  that  the  devices  were  implanted 
intraneurally;  no  longitudinal  data  was  presented  to  show  the 
stability  of  the  device  capabilities  (stimulating  percepts  and 
recording  neural  activity);  and  finally,  decoding  multiple  DOF 
was  not  investigated  [53,  68].  Here,  we  extended  that  work  by 
investigating  multiple  aspects  important  for  developing  future 
bidirectional  neural  prostheses  based  on  high-count  micro¬ 
electrode  arrays,  including:  the  quality  of  percepts  evoked  by 
microelectrode  stimulation,  frequency  modulation  of  percept 
quality,  the  number  of  different  movements  that  can  be 
decoded  using  action  potential  data,  and  the  overall  recording 
and  stimulation  capabilities  of  microelectrode  array  devices 
over  a  30  day  period. 

4. 1.  Stability  of  USE  As  implanted  for  30  days  and  potential  for 
longer  duration  implants 

In  Subject  1-M,  the  impedances  for  the  majority  of  the 
electrodes  went  out  of  specification  at  the  end  of  the  first  week 
due  to  lead  wire  breakage;  however,  four  electrodes  remained 
viable  for  the  duration  of  the  implant.  Suturing  the  percuta¬ 
neous  connector  to  the  skin  appeared  to  have  reduced  the 
chance  of  wire  breakage  in  Subject  2-U  (no  observed  wire 
breakages).  The  SNR  was  stable  after  day  ten  of  implantation 
for  Subject  2-U,  and  we  hypothesize  this  may  be  due  to  the 
stabilization  in  the  surrounding  tissue  as  the  acute  inflam¬ 
matory  response  begins  to  resolve  [69].  Impedances  varied 
significantly  over  the  duration  of  the  study  with  an  increase 
during  post-implantation  days  3-10.  This  variability  may 
reflect  either  changes  in  the  electrode  itself  (surface  chemistry 
of  the  iridium  oxide)  or  changes  in  the  electrode-tissue 
interface  (cellular  milieu).  Furthermore,  impedances  may 
stabilize  after  30  days  as  shown  in  previous  studies  of  USEAs 
implanted  in  the  feline  sciatic  nerve  [70]. 

A  robust  percutaneous  connector  or  a  telemetry  system 
will  be  needed  before  applications  using  microelectrode 
arrays  achieve  clinical  utility.  Also,  the  generation  of  micro¬ 
electrode  arrays  used  in  this  study  have  maximum  electrode 
shank  lengths  of  1.5  mm,  which  limits  the  cross-sectional 
access  to  the  deeper  regions  of  the  relatively  large  human 
peripheral  nerves  that  are  greater  than  3  mm  in  diameter. 
Longer  electrodes  and/or  multiple  arrays  may  be  needed  to 


expand  electrophysiological  access  across  the  entire  diameter 
of  the  nerves. 

A  more  robust  containment  system,  an  anchoring  system, 
or  a  more  compliant  microelectrode  array  design  may  be 
required  to  achieve  the  very  long  functional  lifetimes  neces¬ 
sary  for  clinical  applications.  The  tissue  response  to  indwel¬ 
ling  intrafascicular  electrodes  has  shown  that,  although 
implantation  of  intraneural  electrodes  results  in  tissue 
damage,  viable  neurons  are  found  within  distances 
(<150/tm)  needed  for  safe  stimulation  of  and  selective 
recording  from  neurons  after  >30  days  of  implantation  [70- 
72].  Additionally,  Utah  microelectrode  arrays  have  been 
implanted  for  >5  year  durations  in  motor  cortex  and  continue 
to  record  neural  signals  that  can  be  used  to  control  external 
devices,  suggesting  the  viability  of  this  type  of  interface  for 
longer  duration  implant  times  [10].  Both  subjects  were  given 
dexamethasone  and  minocycline  doses  in  order  to  potentially 
increase  the  quality  of  action  potential  recordings  over  time 
[58,  59].  However,  additional  control  studies  are  needed  to 
investigate  whether  administration  of  these  drugs  can  improve 
neural  signal  longevity  over  long-term  electrode  array 
implantations. 

In  this  study,  action  potentials  were  recorded  across 
multiple  microelectrodes  for  the  duration  of  the  30  day 
implant,  which  validates  that  the  devices  remained  intra- 
fascicularly  implanted  for  the  duration  of  the  study.  More¬ 
over,  electrodes  that  reliably  recorded  action  potentials 
throughout  the  study  generally  evoked  percepts  at  lower  sti¬ 
mulation  amplitudes,  suggesting  that  these  electrodes  were 
intrafascicularly  implanted,  while  other  electrodes  that  did  not 
record  action  potentials  may  have  been  implanted  between 
fascicles. 

4.2.  Neural  recordings  and  decoding  independent  and 
proportional  phantom  finger  movements 

An  important  aspect  of  any  decoded  signal  for  controlling  a 
prosthetic  hand  is  that  it  can  mediate  proportional  control. 
This  was  investigated  in  two  ways.  First,  we  used  a  linear 
Kalman  Filter  to  decode  the  neural  data,  which  provided  a 
continuous  estimate  of  finger  position  that  was  proportional  to 
some  linear  combination  of  the  neural  signals.  Second,  we 
placed  targets  at  various  positions  located  between  full 
extension  and  full  flexion.  This  required  that  the  subjects 
proportionally  modulate  the  neural  signals  in  order  to  acquire 
the  targets  and  complete  the  task.  The  advantage  of  using  a 
virtual  prosthetic  hand  as  opposed  to  an  actual  robotic  hand 
was  programmatic  control  of  target  size  and  distance.  With  a 
virtual  hand,  we  were  better  able  to  quantify  the  accuracy  and 
timing  of  neurally-controlled  movements.  For  both  subjects, 
the  decoded  neural  activity  patterns  provided  proportional 
control  of  finger  position  and,  importantly,  such  control  was 
achieved  during  the  first  recording  session  with  each  subject. 
Some  crosstalk  existed  between  the  DOF,  which  sometimes 
made  it  difficult  for  the  subjects  to  independently  control 
different  digits  during  the  online  experimentation. 

The  relationship  between  the  central  representation  of 
motor  control  and  the  kinematics  of  movement  is  complex, 
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with  activity  in  single  neural  units  correlated  with  the 
movements  of  multiple  finger  muscles  [73,  74].  The  central 
and  peripheral  nervous  systems  control  the  synergistic  bio¬ 
mechanics  of  the  human  hand  [75];  however,  current  high- 
DOF  prosthetic  hands  do  not  implement  mechanical  synergies 
[47,  48].  These  synergies  require  co-activation  of  multiple 
muscles  innervated  by  multiple  nerves.  The  relationship 
between  neural  encoding  and  the  hand  biomechanics  is  fur¬ 
ther  complicated  by  the  possibility  of  post-amputation  neural 
plasticity  and  the  specific  location  of  the  array  implantation 
along  the  proximal-distal  nerve  axis.  Future  work  is  needed 
to  develop  efficient  mapping  of  these  synergistic  neural  sig¬ 
nals  onto  the  non-synergistic  mechanics  of  high-DOF  pros¬ 
thetic  hands,  or  to  develop  prosthetic  hands  that  accurately 
model  the  biomechanics  of  the  human  hand. 

In  this  study  we  correlated  finger  positions  of  the  sub¬ 
jects’  phantom  hand  with  the  neural  signals  recorded  in  per¬ 
ipheral  nerves,  but  did  not  study  the  influence  of  load/force 
or  arm/hand  posture  on  these  efferent  neural  signals.  There 
are  complex  neural  systems  throughout  the  hierarchy  of 
sensory-motor  control  that  will  modulate  the  efferent  neural 
signals  in  the  peripheral  nerve  under  different  loads.  For 
example,  objects  of  different  weights  or  movements  with 
inertial  dynamics  will  require  dynamic  grip  forces  even  under 
isometric  conditions.  Further,  different  postures  of  the  hand 
and  arm  will  interact  with  movement  dynamics  and  require 
differential  modulation  of  the  efferent  neural  signals  for 
similar  finger  positions.  Any  decoding  of  neural  signals  must 
account  for  these  complexities  in  order  to  provide  naturalistic 
control  of  a  high-DOF  prosthetic  hand.  The  development  of 
algorithms  that  decode  neural  signals  from  peripheral  nerves 
for  prosthetic  control  will  continue  to  an  area  of  active 
research.  Improving  the  performance  of  prosthetic  hands  will 
likely  necessitate  the  incorporation  of  afferent  sensory  infor¬ 
mation,  as  well  as  efferent  motor  control  signals,  as  inputs  to 
decoding  algorithms. 

4.3.  Stimulation-evoked  sensory  percepts 

We  explored  the  ability  to  evoke  sensory  percepts  by  injecting 
currents  into  the  peripheral  nerves  via  many  of  the  electrodes 
in  the  implanted  arrays.  In  both  subjects  spatially  discrete 
somatosensory  percepts  were  evoked  using  low  levels  of 
current.  These  spatially  discrete  percepts  could  be  used  for 
registering  contact  of  the  fingers  in  a  prosthetic  hand  with  an 
object  with  high  spatial  fidelity.  Translations  of  non-physio- 
logical  percepts  into  more  physiological  percepts  was 
achieved  by  varying  the  frequency  and  the  duration  of 
microstimulation,  i.e.,  the  percept  changed  from  an  ‘electric 
shock-like  tingle’  to  pressure,  indicating  that  modulation  of 
microstimulation  parameters  may  be  used  to  address  sub¬ 
modalities  of  somatosensation.  Subject  1-M  was  able  to  detect 
and  discriminate  simple  patterns  of  microstimulation,  sug¬ 
gesting  the  more  complex  spatio-temporal  patterns  of  stimu¬ 
lation  could  provide  more  complex  sensation  such  as 
brushing  or  sliding  across  the  skin.  Subject  1-M  noted: 


(1)  ‘As  I  am  pressing  that  down  there  (on  the  manip- 
ulandum  pressure  sensor)  on  that  intensity  and  moving 
the  finger  a  little  bit... this  (the  sensory  percept)  stayed 
on  that  finger  as  I  was  moving  it’. 

(2)  ‘As  they  speed  up  (the  stimulation  frequency  increasing 
from  1  to  100 Hz)  I  can  feel  more  of  the  finger.  ...  It 
applies  pressure  on  the  index  and  this  finger  (of  the 
phantom  hand,  indicated  by  pointing  to  the  tip  of  the 
ring  finger  on  the  subject’s  intact  hand)’. 

(3)  Question  from  the  experimenter;  ‘Could  you  use  this 
stimulation  to  recreate  touch’? 

Answer  from  Subject  1-M:  ‘Definitely... The  more  you 
press  it  (subject  pressed  on  manipulandum  pressure  sensor 
changing  the  frequency  from  1  to  100  Hz)  you  can  sense  it 
(the  phantom  finger)  more.  ...  And  you’ll  get  the  sense  of 
touch,  ‘cause  that’s  what  you  did  for  me’. 

This  subject  was  also  very  accurate  at  reporting  the 
absence  of  any  sensation  when  no  microstimulation  was 
provided.  The  stability  of  the  stimulation-evoked  percepts 
was  assessed,  and  indicated  that  while  evoked  percepts  were 
grossly  stable  they  did  change  over  time.  The  changes  in 
evoked  percepts  likely  results  from  micro-motion  of  the 
electrode  array  relative  to  the  nerve,  and  perceptual  stability 
could  be  increased  by  improving  the  systems  used  to  contain 
the  array  and  anchor  it  to  the  nerve.  The  results  presented  here 
indicated  that  implantations  of  microelectrode  arrays  into  the 
peripheral  nerves  of  amputees  could  provide  sensory  feed¬ 
back  that  would  improve  the  manipulation  of  objects  using 
highly  dexterous  prosthetic  hands. 

Future  studies  are  warranted  to  investigate  the  stimula¬ 
tion  parameters  that  evoke  other  sensations,  such  as  pro¬ 
prioception,  as  well  as,  long-duration  stimulation  to  evoke 
long-lasting  sensations.  Recent  studies  have  shown  modula¬ 
tion  of  stimulation  intensity  with  a  time-variant  pulse  width 
results  in  more  natural  evoked  perceptions  [45]  and 
employing  these  methods  may  improve  the  quality  of  percepts 
produced  by  microelectrodes.  Moreover,  Tan  et  al  have  also 
demonstrated  stability  of  percept  location  may  stabilize  after 
27  weeks  post  implantation [56],  and  thus,  longer  duration 
USEA  studies  are  warranted  to  assess  such  time  periods. 

4.4.  Post-stimulation  evoked  sensory  percepts 

The  subjects  experienced  post-stimulation  percepts  that 
occasionally  had  the  uncomfortable  or  painful  qualities  of  the 
percepts  evoked  by  some  electrical  stimulation.  Importantly, 
neither  of  the  subjects  experienced  these  qualities  of  percepts 
prior  to  participating  in  the  study.  Future  work  should  address 
optimizing  microstimulation  parameters  to  produce  only 
physiologically  relevant  sensory  percepts,  and  investigate 
plasticity  in  the  central  motor  and  sensory  neural  repre¬ 
sentations  resulting  from  long-term  microstimulation  of  the 
peripheral  nerves  [76-80].  Subject  2-U  noted  the  feeling  of 
the  non-painful  post-stimulation  percepts  in  the  subject’s 
phantom  limb  sensation  diary: 

‘I  have  had  the  kind  of  fluttering  or  breath-on- 
the-skin  or  hair-pressure  that  I  had  often  in  the 
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lab  session  today.  (Located  in  the  web  area 
between  my  PIF  (phantom  index  finger)  and 
PMF  (phantom  middle  finger).)  It  is  as  if  now 
that  the  sensation  has  been  awakened,  it  keeps 
registering,  regardless  of  context — a  little  like 
a  kid  using  a  new  vocabulary  word  liberally  or 
a  cook  using  a  favourite  spice  or  herb  in  all 
kinds  of  meals...’ 

These  ‘awakened’  phantom  limb  sensations  may  have 
implications  for  embodiment  of  neural  prostheses.  Consistent 
stimulation  of  the  nerve  may  engage  or  reactivate  central 
neural  circuits  that  encode  the  representation  of  the  phantom 
or  prosthetic  limb.  This  embodiment  of  a  prosthetic  limb 
could  be  enhanced  by  coupling  the  nerve  stimulation  with 
simultaneous  input  from  other  sensory  modalities,  e.g.  vision 
of  the  limb  being  touch. 

5.  Conclusion 

Numerous  electrodes  on  the  arrays  recorded  neural  activity 
patterns  from  residual  nerves  that  were  volitionally  generated 
by  the  intention  to  flex,  extend  or  abduct  individual  digits  in 
the  subjects’  missing  hand.  Up  to  thirteen  movement  types 
could  be  decoded  offline,  and  proportional  control  of  up  to 
two  digits  of  a  virtual  prosthesis  was  achieved  online.  Sti¬ 
mulation  of  up  to  96  electrodes,  either  one-at-a-time  or  via 
small  groups  simultaneously,  evoked  multiple  percepts  that 
were  spatially  distributed  across  the  phantom  hands  in  ana¬ 
tomically  appropriate  distributions.  The  relatively  large 
number  of  channels  of  motor  and  sensory  information  pro¬ 
vided  by  the  microelectrode  arrays  indicate  that  such  arrays 
can  serve  as  a  neural  interface  for  controlling  high-DOF 
prosthetic  limbs.  The  subtle  verbal  descriptions  of  evoked 
perceptions  from  the  subjects  indicated  an  attempt  to  integrate 
the  sensations,  either  cognitively  or  perhaps  due  to  neural 
plasticity,  into  their  subjective  bodily  representation.  Patients 
outfitted  with  a  highly  dexterous  prosthetic  limb  controlled 
through  such  a  bi-directional  peripheral  nerve  interface  might 
begin  to  think  of  the  prosthesis  not  as  a  piece  of  hardware 
attached  to  their  arm,  but  rather,  as  an  integral  extension  of 
themselves. 
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